Skip to content

fix(rabbitmq): wait the broadcast defer delay before requeueing a deferred message - #217

Merged
zannis merged 1 commit into
mainfrom
zannis/fix/broadcast-defer-ordering
Sep 26, 2026
Merged

zannis merged 1 commit into
mainfrom
zannis/fix/broadcast-defer-ordering

Conversation

@zannis

@zannis zannis commented Sep 25, 2026

Copy link
Copy Markdown
Owner

Summary

A broadcast Defer is retried in place on every backend, and Kafka, NATS, Redis and InMemory wait BROADCAST_DEFER_DELAY (1 s) before handing the message back. RabbitMQ's broadcast router nack-requeued immediately, so a handler that keeps deferring was redelivered at wire speed on this backend only.

  • route_broadcast_outcome's Defer arm now waits BROADCAST_DEFER_DELAY before nack_requeue, raced against the subscription's shutdown token so a drain is not held open. The wait holds the single delivery slot (broadcast pins prefetch to 1), so the requeued message is still handed back before the message queued behind it.
  • BROADCAST_DEFER_DELAY's re-export is now compiled under rabbitmq as well.

Tests

  • New rabbitmq_broadcast_integration::defer_waits_the_defer_delay_and_redelivers_in_place: asserts order a, a, b and a gap of at least 1 s between the two a calls. It failed before the fix (redelivered after ~0.8 ms).
  • New inmemory_broadcast::a_deferred_message_waits_the_defer_delay_before_redelivery: pins the delay InMemory already applied; the existing ordering test would have passed without the wait.

Test plan

  • cargo nextest run --features rabbitmq --test rabbitmq_broadcast_integration (5 passed)
  • cargo nextest run --features rabbitmq,rabbitmq-transactional,metrics --test rabbitmq_integration (82 passed)
  • cargo nextest run --features inmemory,metrics,sbe,env-config (807 passed)
  • cargo fmt -- --check; cargo clippy --all-features --all-targets and --no-default-features --all-targets with -D warnings; cargo clippy --lib --no-default-features --features <each backend> with -D warnings

…erred message

Kafka, NATS, Redis and InMemory hand a deferred broadcast message back
after BROADCAST_DEFER_DELAY. RabbitMQ nack-requeued it at once, so a
handler that keeps deferring was redelivered at wire speed on this
backend alone. The Defer arm now waits the shared delay, cut short by
shutdown, before the nack-requeue. The wait holds the single delivery
slot, so the message still comes back ahead of the one behind it.

Tests pin the delay on RabbitMQ and InMemory; the existing ordering
tests did not, and would have passed with the wait removed.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
shove-docs 0a0d4ec Commit Preview URL

Branch Preview URL
Sep 25 2026, 11:22 PM

@codecov

codecov Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/backends/rabbitmq/consumer.rs 90.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@zannis
zannis merged commit 022e36e into main Sep 26, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant